DOCUMENT RESUME 

ED 352 721 EA 02A 528 



AUTHOR 
TITLE 



INSTITUTION 
SPONS AGENCY 
PUB DATE 
CONTRACT 
NOTE 
PUB TYPE 



Herman, Paul; And Others 

Preliminary Data Analysis of Characteristics 

Change Agent Projects. Study of Change Agent 

Programs: A Working Note. 

Rand Corp., Santa Monica, Calif. 

Office of Education (DHEW) , Washington, D.C. 

Sep 7A 

WN-8828-HEW 

133p. 

Reports - Research/Technical (iA3) 



of 



EDRS PRICE MF01/PC06 Plus Postage. 

DESCRIPTORS '^Educational Change; '^Educational Innovation; 

Elementary Secondary Education; ^Federal Aid; 
''Tederal Programs; Models; Program Implementation; 
^School Districts 



ABSTRACT 

Preliminary findings of a study that identified 
characteristics of federally funded change agent programs are 
presented in this paper. The study examined four federal change agent 
programs: Elementary and Secondary Education Act (ESEA) Title III, 
Innovative Projects; ESEA Title VII, Bilingual Projects; Vocational 
Education Act, Part D, Exemplary Programs; and the Right~to-Read 
Program. Data from a national survey of 225 projects in 18 states and 
case studies of 30 projects were analyzed to determine which factors 
systematically affect the change process. Project designs were 
categorized by goals, educational approach, change strategy, and 
resources. Independent variables of the institutional 
setting — district size, wealth, expenditure per pupil, source of 
financial funding, community demographics, and the 
superintendent — were analyzed for their effects on a school 
district's "innovat iveness . " The districts most likely to adopt 
innovative programs were larger; had higher density school 
enrollments, adequate budgets, higher family incomes, and more 
experienced superintendents; and were less dependents on state aid. 
Thirteen tables are included. The appendix contains methodology notes 
and eight statistical tables. (Contains five references.) (LMI) 



* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 





WN-8828-HtW 



September 137^ 



PRELIMINARY DATA ANALYSIS OF 
CHARACTERISTICS OF CHANGE AGENT PROJECTS 

Paul Berman, Edward Pauly, 
Roger Rasmussen, Sinclair Coleman, ^ 
Francois Christen and Cathleen Vickers ^ 



U.S. OCPAfTTMCNTOP EOUCATrON 
OHtc« of Educational Ptsetrch and improvement 

National resources information 
center (eric) 

^This document r.as txen reproduced as 
received from the person or organization 
originating it 
C Minor changes have t>een made to improve 
reproduction Quality 

• Points of view or opinions stated m t hia docu- 
ment do not necessarily represent official 
OERi position or pohcy 



WN-8828-HEW 
September 197^ 



PRELIMINARY DATA ANALYSIS OF 
CHARACTERISTICS OF CHANGF. AGENT PROJECTS 

Paul Berman, Edward Pauly, 

Roger Rasmussen, Sinclair Coleman, 

Francois Christen and Cathleen Vickers 



iii 



PREFACE 



Rand is conducting, under the sponsorship of the U,S. Office 
of Education, a two-year study of federally funded programs designed 
to introduce and spread innovative practices in public schools. 
These change agent programs normally offer temporary federal funding 
to school districts as "seed money." If an innovation is successful, 
it is assumed that the district will continue and disseminate part 
or all of the project using other sources of funds. The Rand 
study examines four such federal change agent programs — Elementary 
and Secondary Education Act Title III, Innovative Projects; Elemen- 
tary and Secondary Education Act Title VII, Bilingual Projects; 
Vocational Education Act, Part D, Exemplary Programs; and the Right- 
to-Read Program. The study identifies what tends to promote various 
kinds of changes in the schools and what doesn't; in particular, the 
Rand study will identify for federal, state, and local policymakers 
the quality, permanence, and extent of dissemination of innovations 
that are associated with the various federal programs and with 
various federal, state, and local practices. 

A series of reports will describe the results of the first 
year of the Rand study (July 1973-July 1974). Volume I (R-1589, 
A Model of Educational Change) will provide a theoretical perspec- 
tive for the Rand study by analyzing the current state of knowledge 
of planned change in education and by proposing a conceptual model 
of factors affecting change processes within school districts. 

Volume II of the series (R- , CJiavaatevistics of Change 
Agent Projects) will contain the analysis of survey data collected 
by a national sample of 225 projects in 18 states during November 
and December 1973. 

Volume III (R- , The Process of Implementing Change) sum- 
marizes the results of 30 case studies of change agent projects 
conducted by Rand staff members and consultants in 25 school 
districts during April and May 1974. These case studies were 
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chosen from the original sample of 225 projects initially survived. 
Volume III also describes the role of state education agencies in 
choosing and disseminating the change agent projects. 

Volume IV (R- , Synthesis of Findings) summarizes the 
findings of Volumes I, II, and III, and also synthesizes extensive 
data collected by Rand on federal level program strategy and manage- 
ment for each of the change agent projects. Volume IV also includes 
a discussion of alternative federal strategies for promoting inno- 
vation • 

There will also be an executive summary volume which presents 
a summary of the study's methods and results for a general audience. 
Finally, there will be two technical appendices, one containing 
brief summaries of each of the 30 case studies analyzed in Volume 
II, and the second including a detailed description of the genesis, 
innovation strategies, and management styles of each of the federal 
change agent programs analyzed in this study. 

The second year of the study will collect additional data on 
Titles III and VII of ESEA., with particular focus on projects whose 
federal funding has expired. The final report of the second year's 
work will be issued in December, 1974. 

This Working Note outlines the progress made on the data 
analysis for Volume II. Both the results discussed and the approach 
indicated are preliminary and will be refined, elaborated on, and 
extensively added to for the final report. The purpose of presenting 
these early findings is to provide a means for eliciting comments 
and suggestions for subsequent analysis. For a detailed outline of 
Volume II, see Revised Data Analysis and Reporting Plan^ WN-8754-HEW, 
July 1974. 
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1. INTRODUCTION 



A major objective of the Change Agent Study is to identify for 
federal, state, and local policymakers the quality, permanence and 
extent of dissemination of innovations in local school practices that 
can be expected for specified combinations of project characteristics 
and institutional settings. To accomplish such an ambitious objective 
requires a variety of overlapping data collection and analysis techniques. 
This Working Note presents a preliminary statement about the quantitative 
analysis efforts of the Change Agent Study. We will describe the basic 
approach in general terms and indicate detailed aspects of the analysis 
to date. At this early stage, it would be premature to offer policy^ 
relevant interpretations of our preliminary iiLvestigations. Accordingly, 
we refrain from such interpretations; rather we present pieces of the 
analysis so that the reader may better understand the direction of the 
inquiry. 

AN OVERVIEW 

The ultimate objective of federal policy in the area of inno- 
vative projects is to improve the education of children by introducing 
changes into the educational process. Volume I of this series 
argued that the relationship between federal policy and the desired 
objective cannot be systematically formulated without analysis of 
the processes of change within the school district. 

In particular, the process of change begins with the introduction 
of an innovative project and goes through the following three stages: 

(1) support 

( 2 ) imp lemen t a t i on 

(3) incorporation 
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The support stage is that initial period in the life of an innovative 
project when plans are conceived and formulated, money and resources 
are sought, and decisions are made by the local school officials as 
to which projects they should select and back. In the second stage, 
the project confronts the reality of its institutional setting and 
implementation begins. In many projects — particularly those that 
ultimately cause significant change in educational practices — implex- 
mentation involves a process of mutual adaptation in which the initial 
characteristics of the project are adapted to the school and class 
environment, and the teachers, principals, and other relevant actors 
adapt their behavior to the requirements of the project. The final 
stage involves either the incorporation of the project (in part or 
whole) into the standard practice of the district or its demise. 

The data analysis will attempt to determine which factors syste- 
matically affect the change process sketched above. Figure 1 presents 
a schematic diagram of those factors that the literature suggests play 
a major role in the innovative processes. Using this conceptual model 
(which is discussed in detail in Volume I), the analysis will operation-- 
alize many of the concepts indicated by Figure 1 and investigate their 
interrelationships by means of statistical techniques. In particular, 
we will measure, describe, and characterize 

o project characteristics 
o institutional settings 

o project outcomes in the areas of success, implementation, 

behavioral change, and continuation 
o federal and state policy inputs 

Moreover, we will estimate, and test hypotheses about, the extent to 
which the various project characteristics affect project outcomes given 
different institutional settings and federal and state policy inputs. 
All measurements to be discussed below are based upon a nation- 
wide survey of a sample of 289 innovative projects and intensive 
field work in 30 projects. The survey, which was administered by 
the National Opinion Research Center in approximately 200 school 
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districts in 18 states, consists of a multi-- level series of ques- 
tionnaires. Superintendents, federal program managers , project 
directors , principals , and teachers were interviewed and asked to 
provide information and express opinions about the characteristics 
of these projects, about the innovative process, about their role 
in this process, and about the federal and state role in fostering 
innovation and dissemination. In addition to the survey data, 
quantitative data drawn from such other sources as census data, OE 
files, ELSEGIS data files, and interviews with state educational 
agency officials are used. The appendix to this report examines 
the representativeness of the survey and field work sample. 

The data were collected for different 7e^els in the school 
organir^ation because we believe that decisions, processes, and the 
type of information available vary with the level involved. For 
example, we expect that the superintendent and school district 
officials play major roles in the support and incorporation stages 
but often do not directly affect the implementation stage. Accordingly, 
our analysis is composed of two phases—that of the implementation 
of the project involving teachers, principals, and project directors, 
and that of the support and incorporation stages involving higher 
school officials. 

Since implementation necessarily depends on the way teachers 
translate project design characteristics into reality in the classroom, 
the measurements, descriptions, and statistical analysis of implementation 
focuses on teachers and how they respond to the project. In symbolic 
terms, we will be estimating equations of the following form at the 
individual classroom level: 

Teacher Implementation Outcome = f (Project Ch?racteristics , 
Institutional Setting, Federal and State Programs and Policies) 

The implementation outcome will be measured by the responses of teachers 
(n==690) to questions about success, difficulty of implementation, the 
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extent to \^ich the project was implemented as originally laid out, 
and the extent to which teachers believed they changed their 
behavior as a result of the project. Section II of this Working 
Note describes our progress in the exceedingly-difficult task of 
characterizing the projects. Volume II will treat the measurements of 
the other variables and present the estimations. 

The second phase of the analysis will deal-insofar as possible — 
with the school district level and questions of adoption, continuation, 
and dissemination. The reason for this second phase is that (a) for 
federal and state policymakers, a highly-relevant policy outcome 
concerns the school district's decision to commit local resources to 
the continued support of a "successful" project when federal funds 
expire and (b) the superintendent's evaluation of a project and his 
willingness to commit resources to it depends only weakly — according 
to our preliminary analysis — on how "successful" the project imple- 
menters perceive the project to be. In short, we will consider the 
"implementation outcomes" of the project to be an input to the decision 
processes involving continuation and dissemination. In symbolic terms, 
we will estimate equations of the following form at the school district 
level (n^l90): 

Incorporation and Perceived Success = f (Perceived Implementation 
Outcomes, LEA Characteristics, Project Characteristics, and Federal 
and State Policy) 

Since progress on this second phase depends upon completion of the analysis 
of the implementation phase, this Working Note can present (in Section III) 
only one piece of the school district analysis — namely, the characteristics 
of the LEA that we call innovativeness. 
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II, PROJECT CHARACTERISTICS 

One of the reasons why the range of diverse innovative projects 
tried at the local school district level is so hard to characterize 
is that different dimensions are implicitly involved in their design. 
Therefore, to describe these projects systematically, we find it con- 
venient to categorize any project into its characteristics on four 
components of project design: 

(1) focus or goals 

(2) educational method or approach 

(3) change, or implementation, strategy 

(4) resources 

The focus, or goals, of the project is the collection of objectives 
of the innovation — be they highly specific, such as "increased reading 
scores of under-achievers in grades 3-6," or highly diffuse, such as 
"improve the humanistic learning of children," The educational method 
or approach is the specific technique applied in the teaching process 
to reach the goals — e.g., an open classroom or a SWRL readivxg method. 
The change strategy is the group of activities employed by the project 
to implement the educational method — e.g., the amount of staff training. 
Finally, the resources specify the scope, scale, and intensity of the 
effort. 

FOCUS 

Determining the focus of innovative projects has proved to be an 
extraordinarily difficult task. Even ignoring for the present analysis 
the personal and sometimes "hidden" goals of individuals involved in 
the project, the catalogue of project objectives is long and highly 
diverse. For such federal programs as Right- to-Read, there are osten- 
sible common goals that, however, tend to subdivide into rather 
different specific objectives. For State Title III and Federal Title III, 
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no requirement of common purpose exists and the projects are accordingly 
particularistic. Nonetheless, systematic comparisons require that pro- 
ject goals be abstracted and grouped. Table 1 lists a categorization of 
the primary focus of the innovative projects in the Rand sample. 

This categorization was formulated by means of a content analysis 
of titles, abstracts, and other available information about individual 
projects in the Rand sample. Of course, most projects could be described 
by more than one category — e.g., reading projects could be targeted 
towards special problems in the school. However, we believe that many 
projects have a raison d'etre implying a dominant focus (of the type 
listed by Table 1) from which design specifications follow. 

Table I categorizes the primary goal or focus into four major 
divisions — curriculum changes, school changes, targeted change, and 
extra services. 

Curriculum Changes 

Many educational innovations are oriented towards changing curriculum. 
In particular, there were four major areas of curriculum change. First, 
there were projects that aimed to enrich the curriculum in the sense that 
a new class or subject of instruction was added onto the existing core 
curriculum. The following substantive foci were identified in this enrich- 
ment category: 

o Environment 

o Career Education 

o Drug Abuse Education 

o Music or Art 

o Physical Education 

o Drop-out Information 

o Ethnics and Humanistic 

Another collection of projects focused specif ically. on improving the 
reading curriculum. Some of these innovations dealt with remedial 
reading others adopted pre-designed reading packages or technologies . 
Third, some projects were mathematics improvement programs that 

13 
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Table 1 

PRIMARY FOCUS OF INNOVATIVE PROJECTS 



Curriculum Changes 
Enrichment 
Reading 
Mathematics 
Language Development 

School Changes 
Classroom 
Organi zation 
Technology 
Staff Training 
Parents Involvement 

Targeted Change 

Coping with Special Problems or Needs 
Cultural Groups 

Extra Services 
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spanned the range from computer-assisted instruction to the use of 
the Nuffield approach, A final group of pro3ects concentrated on 
development of language skills. 

School Changes 

A number of innovations apparently derived their design require- 
ments from ideas about how and what to change in the schools. Five 
different foci of change emerged in the Raad coding. First, there 
were such classroom change projects as open classrooms and peer in- 
struction. Other innovations aimed beyond the classroom to the 
larger organization of the school or school district — e.g., alterna- 
tive schools. Another group of projects introduced new technologies 
involving hardware — e.g., television or computers — into the life and 
educational repertoire of the school. Some innovations involved the 
explicit training of staff to new ways that could change their usual 
roles and behaviors. Finally, a number of projects focused on 
bringing parents into an active relationship with the. school. 

Targeted Change 

Targeted change represents another category of goals pursued 
by these projects. In particular, some innovations were designed to 
cope with special problems or needs in a school or district — e.g., 
project's ranging from Harlem Prep to handicapped -children programs. 
In addition, "cultural groups" — e.g., Spanish-speaking students or 
American Indians — were targeted. 

Extra Service 

A final category involved programs providing such extra services 
to the classroom, school, or district as resoui'ce centers or library 
facilities. 

EDUCATIONAL METHODS 

The designers of projects at the level of the local school 
district may not, and often do not, plan in neat analytical terms that 
separate ireans from ends. Rather they begin with a set of ideas 
consisting of an intermixture of goals ^ techniques ^ and strategies. 
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Nonetheless, we need to separate these aspects of projects. Accord- 
ingly, this section suggests an analytical classification of the 
educational method or approach of the innovations in the Rand sample. 

We asked project directors of 289 projects to check off the 
educational techniques employed in their project. Table 2 presents 
marginal results. A glance at the numbers of techniques mentioned 
suggests that most projects used several methods in combination. 
The possible combinations of the twenty distinct methods are very 
many indeed. However, statistical analysis shows that various 
educational techniques tend to be associated with each other so that 
the twenty techniques may be grouped into a small number of dis- 
cernible patterns. Table 3 displays the results of a factor analysis 
designed to discriminate the smaller number of patterns.* 

The following five analytical types of educational methods 
emerged froiu the factors analysis: 

• Behavioral modification techniques involving such methods 
as student performance incentives and various technological 
innovations . 

• Enrichment programs involving heavy components of field 
trips and community resources and clearly not involving 
diagnostic methods. 

• Classroom organization methods typified by open classrooms, 
non-graded or ungraded classrooms, and team teaching. 

• Intensive staffing of traditional teaching approaches. 

• Organizational changes in the school such as new management 
techniques or the adoption of new curriculums. 

In addition to the value of classifying innovative projects as 
varying along each of the dimensions presented above, the factor 
analysis also produces for every project a series of five factor 
scores that allow any project in the Rand sample to be measured in 
terms of its particular mix of educational methods. Subsequent 
analysis makes use of these analytical scores. In particular. 
Table 4 presents a summary comparison of how these analytical 

*A discussion of the statistical analysis is not included in 
this preliminary report. 
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project characteristics are distributed among the several federal 
programs. The average values of the factor scores suggests that 
the Vocational Education programs primarily deal with enrichment 
methods; the Right- to-- Read projects rely on intensive staffing using 
traditional methods and behavioral modification techniques; the Bilingual 
projects do not involve school administrative changes but concentrate 
on a combination of intensive staffing, enrichment, and some class- 
room organization changes. Due to the extraordinary diversity of 
Title III projects, the factors show a heterogenous distribution 
for projects funded by State or Federal Title III, (Table 5 A-E 
presents a more detailed breakdown of the summary figures in 
Table 4,) 
CHANGE STRATEGY 

The design of innovative projects explicitly, or more usually 
implicitly, designates a strategy for implementing the change antic- 
ipated by its goals and educational methods. The survey instruments 
used in this study collected data about those various elements of 
change strategies identified in the literature as being effective in 
aiding the implementation of innovations in the school environment* 
In particular, the following elements were measured: 

o planning 

o staff training 

o project meetings 

o actor participation 

o support by principle actors 

o implementation flexibility 

o incentives offered teachers 

o amount of change or effort required of teachers 
o selection of schools and teachers 

During the course of the analysis , each of the above elements 
will be examined to determine its infiU(r;nce xipon project outcomes. 
For example, statistically contolling for other relevant variables, 
we will estimate the extent to which planning affected the implemen- 
tation and the perceived success of projects. To give the reader a 
more detailed impression of how the various elements of change 
strategy were measured, the following list catalogues some of the 
operational items to be used in the analys ) 7 
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Table 2 

EDUCATIONAL METHODS OF PROJECTS 

Special projects ocxn call fov a ^^ariety of 
diff event techniques. FZease tell me which 
of the techniques on this card the project 

makes use of. CODE ALL MENTIONED. Mentioned 

Needs assessment c c • • 224 

Para-professional staff (teacher aides, etc,) 188 

Instructional specialists ...o 180 

Counseling and guidance specialists 90 

Performance incentives for students 100 

Educational technology (audio -visual materials , 

computers, etc.) 186 

Development of new curriculum or materials 225 

Adoption of new curriculiim 113 

Field trips 141 

Open classrooms 88 

Non-graded or ungraded classrooms 81 

Learning centers « r. 169 

Peer instructions 130 

Team teaching 148 

Individualized instruction 234 

Behavioral objectives 216 

Diagnostic and prescriptive methods 181 

New management techniques (decision-assisting technology 

like PPBS, MBO, etc) 81 

Parent involvement 201 

Use of community resources • 185 

Total Sample = 289 
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Planning - Number of months spent in planning and writing the 
original application; percent of project's first year budget 
spent in planning and project design; whether the methods or 
materials of the project were developed at site or elsewhere. 

Staff training - Percent of the project budget allocated to 
training; the proportion of the teachers on a project re- 
ceiving training; the amount of time participating teachers 
spent in training. 

Project meetings - The frequency with which principals meet 
with project staff; the frequency of project meetings; the 
teacher's assessment of the value of the meetings. 

Actor participation - The extent to which teachers and prin- 
cipals participated in various project decisions. 

Support by principle actors - The extent to which project 
directors received support from the superintendent, the 
federal program manager, principals, faculty, and funding 
agency personnel. 

Implementation flexibility - The extent to which teachers, 
principals, and project directors had freedom to alter 
project design characteristics during implementation. 

Incentives offered teachers - Whether teachers received 
extra pay for training. 

Amount of change or effort required of teachers. 

Selection of schools and teachers - How schools were chosen; 
how teachers were chosen; the proportion of teachers at 
school involved in a project. 



RESOURCES AND FEDERAL PROGRAMS 

The third element characterizing the projects is resources. Of 
course, the resources received by the projects uiider consideration in 
this study depend upon federal programs and their policies. This 
section briefly describes the projects in the Rand sample in terms of 
resources and federal programs . 

Information describing the sampled projects can be grouped into 
three classifications: policy data, methods data, and individual- 
project-history data. The amount of resources applied to a project 
is related to these classifications, as will be shown. 

Policy data are information on the explicit goal selected by 
authoritative decision-makers at all levels for projects. It in- 
cludes information on the particular federal grant-in-aid program 
which provides funding for each project, and the primary policy focus 
selected by each LEA for its project. Some federal program — Right- 
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to-Read, Bilingual (Title VII), Federal- and State-Administered 
Vocational Education — are explicit in their policy focus. Title III 
programs are diverse in focus, but some state and federal policy-makers 
have suggested or required applicant LEA's to focus on one or more 
areas. For example, the Office of Education established seven pro- 
gram priorities for its Section 306, Title III program. Potential 
applicants were encouraged to submit proposals within these sug- 
gested '^areas of national concern*': 

early childhood education, 
environmental education, 
education of the disadvantaged, 
reading , 

drug abuse education, 

human diversity and cultural pluralism, and 
student and youth activism. 

This list of priorities was intended by the Office of Education to 
guide and encourage local decision-makers, rather than to describe a 
universe of acceptable project types. In fact, projects were even 
more diverse than the guidelines suggest: the priorities were mod- 
ified (or at least increased in number) in the second year of the 
Section 306, Title III program; the applications received for the 
"human diversity" and "student activism" categories were very small 
in number; and applications were received — and subsequently funded — 
which were associated with none of the "priority" areas. Moreover, 
the categories are of little use for describing the projects. It 
appears that descriptively grouping the innovative projects by the 
USOE policy categories (which were intended to focus those projects 
in particular areas) is highly vulnerable to problems of likely 
variation within categories, instances of virtually empty categories, 
non-comparability of the categorization across several years of 
different policies, and lack of comparability with categories used 
for other federal programs. 

Another problem with descriptive data based on policies used by 
program decision-makers is illustrated by the state Title III guide- 
lines used by the California SEA. Applying LE^s were required to 
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use Title III funds strictly for the development or improvement of 
reading and mathematics programs. That is, any application submitted 
for California's state Title III funds had to show its relevance to 
the goal of improving reading or mathematics performance by students. 
While it may be reasonable for us to ask project directors whether 
their project focuses on reading, mathematics, or other student needs, 
the California constraint may have the effect of causing projects 
which focus on "attitude change" or "environmental education" or 
"cultural pluralism" or "dropout prevention" to he described as vead^ 
ing projects — therefore, using a simple reading/math classification 
question for California projects could yield misleading results. 

Another sort of policy information about Change Agent projects 
might be the declared goal of the project. So long as project goals 
are either set by LEAs or by federal guidelines, we may obtain an 
idea of the aim of each project. Several federal programs have fimd- 
ing guidelines which are quite restrictive of project goals: 

Title VII, ESEA 36 projects in sample 

Right -to -Read 32 projects in sample 

Vocational Education, Part D 

(State administered) 15 projects in sample 

Vocational Education, Part D 

(Federally administered) 9 projects in sample 

For these federal programs, it seems reasonable to assume that there 
is comparatively little internal variation in program goals. For 
the more varied state Title III and Section 306 programs, project 
directors were asked "to pick one primary focus of the project," and 
responded as follows: 

Title III Primary Focus Projects in Sample 

Bilingual Education 6 

Career Education 8 

* 

Although half the federal Vocational Education programs were 
made part of Model Cities projects, and 'Wtated" as a result. 
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Individualized or diagnostic/prescriptive 

reading curriculum 39 

Individualized or diagnostic/prescriptive 

curriculum other than reading 26 

Classroom organization changes 27 

Enrichment classes or activities 9 

Improving district planning or 

management practices 10 

Something not listed 66 



Comparing the federal grant-in-aid programs and the focus of Title 
III projects, we obtain a description of the Change Agent sample of 
projects which shows the diversity of goals found in the sample, 
[See Table 6,] Note that there are sampled Title III programs which 
correspond (in focus) to sampled programs funded by federal and state 
Vocational Education grants, Right-to-Read grants, and Title VII 
grants. This correspondence may make it possible to analyze the 
impact of different federal program guidelines on the outcome of 
innovative projects, A major lesson which we take from a review 
of the policy data is that a fixed level of resources will have 
different effects when applied in different policy areas. 

Methods data include information on the selected approaches 
adopted by LEAs for solving their special educational problems. The 
particular method or technique applied by LEAs — whether it is the 
creation of learning centers, or staff development in the use of 
behavioral objectives, or the planning of a new curriculum need not 
be directly related to the policy focus of the project. For example, 
staff development could be the primary method used in bilingual, 
career education, or reading improvement projects. 

Indirect information on some aspects of the projects' methods 
can be obtained from project-specific data on the targets of the 



A major problem with the analysis of this sort of methods data 
is that educational-change projects frequently combine several 
methods. If projects are to be properly characterized, it would be 
desirable to identify underlying dimensions of methods rather than 
the simple presence or absence of a given method. But since the 
underlying dimensions of methods may not be known to project direc- 
tors, considerable analysis must be used to develop them. The 
earlier section of educational methods illustrates an analytical 
means for characterizing projects. 
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innovation. The types of schools (elementary, high school, and so 
forth) and number of students served by the project can be used to 
understand the level, extent, and concentration of the Change Agent 
projects ♦ 

Many projects serve more than one type of school. For example, 
of the 104 sampled projects serving high schools, 65 also serve ele- 
mentary schools* We can categorize some of the combinations of 
types of schools served by Change Agent projects: 

Types of Schools Served Number of Projects 

Both elementary and high schools, or 

both elementary and junior high 97 

Elementary schools, but no high schools 
and no junior highs and no out -of -school 
or adult programs 106 

High schools and/or junior highs, but no 

elementary schools or pre-schools 57 

Other combinations of school types, 

including* e.g*, exclusively non-public 

or exclusively out-of -school and 

adult programs 18 

Management practices, staff characteristics, the amount of flexibility 
required in dealing with individual student problems, and the kinds of 
expertise relevant to the process of innovation may be expected to 
vary across these types of schools. It seems likely that LEA staff 
responsible for implementation of new projects are quite aware of the 
kind of school they are dealing with, and that they act on this 
knowledge . 

A second bit of indirect evidence on the project's likely type 
of approach to innovation is the number of students it serves. The 
range of target group sizes for Change Agent projects is large: from 
fewer than 50 to more than 5,000 students may benefit from a single 
project. The frequency distribution of the size of projects in the 
Rand sample is given in Figure 2. A good deal of the variation in 
project size is associated with scliool district size (that is, with 
total enrollment), and consequently with urbanness. But there are 
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small projects even in cities, and a wide variety of policy goals 
have been attempted for projects having many different sizes. 

Individual-project-history information refers to data describing 
the administrative decisions made in the course of introducing and 
implementing the project. Two such kinds of information are the year 
in which each project was initially funded, and the level of funding 
which it achieved. The year of initial project funding reflects at 
least two major facts about the project's history — the external cir- 
cumstances (including federal and state policies) at the time the 
project was designed and implemented, and the amount of elapsed time 
from the project's initiation to the time of data collection (1974). 

Though the sample of Change Agent projects does not allow us to 
infer population characteristics for all school districts receiving 
federal grants, it seems that the six federal grant-in-aid programs 
have had different histories. Table 7 gives the distribution of 
sampled projects by federal program and year of first funding. While 
most projects have been in existence for two, three, or four years, 
no Right-to-Read projects could have been begun before 1971, and 
other programs also have "lumpy" distributions for historical reasons. 
(The bulk of Section 306 projects were begun in 1971, and new funds 
were appropriated at much smaller levels after that year.) 

Table 7 also enables us to see that 47 projects had been in 
existence four or more years when the data were collected, 112 had 
been in existence three years; and 110 had been in existence one or 
two years. Analyzing the distribution of Title VII (Bilingual) pro- 
jects across years, for example, may allow us to judge whether dif- 
ferent problems of implementation, or perhaps different perceptions 
of success, apply to projects with different lengths of implementa- 
tion time. That is, both reported events which affect the project, 
and outcomes of the project, may depend on how long it has been in 
place in a school system. 

While federal grants-in-aid are generally designed to bear the 
greatest share of the increased costs associated with innovative 
school programs, they are not the only source of funds for innova- 
tion. Many Change Agent projects obtained substantial funds in 
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addition to their main federal grant, from state categorical programs, 
special LEA appropriations, or foundations; also, some LEAs have 
"packaged" federal grants from, such sources as Title I, EPDA, or NDEA 
to supplement the primary Change Agent grant. For our evaluation of 
the effect of extra resources on project outcomes, we should use in- 
formation on the sum of special grants from all sources which support 
the innovative project. But to analyze the "policy-level" effect of 
the size of federal Change Agent allocations, we should depend only 
on the main grant. 

The problem of which grant total to analyze is made more complex 
by the fact that most projects have had considerable variability in 
funding over the several years they have existed. What is the best 
summary of the "fiscal scope" of a project which received $40,000 in 
its first year, including a $10,000 planning grant; $125,000 in its 
second year; and $75,000 in its third year? Clearly the variability 
and direction of trend in funding can be used as indicators of partic- 
ular federal or SEA policies on planning, project development, and 
encouraging continuation. For our purposes of describing the scope 
of the projects, however, even the average grant received over 
several years is somewhat deceptive, because of year-to-year varia- 
tion; for example, a project primarily concerned with introducing 
new technology and hardware to a school may have a large capital 
grant in one year, and much smaller support and maintenance grants 
in other years. A useful summary of a project's funding which focuses 
on the resource requirements of the project at its peak is the value 
of total of special project grants in the year of greatest expenditure. 
The frequency distribution of this measure of project funding is shown 
in Figure 3. While quite a few Change Agent projects depended on 
federal grants which in any one year never exceeded $75,000 (about 90 
of them), most projects had at least $100,000 in grant funds avail- 
able for implementing an innovative project. The distribution of 
total grant packages is highly skewed — that is, there are many 
projects whose size is less than $150,000, and there is a slow, 
steady falling-off of project frequencies as the grants grow very 
large. 
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By combining our information on the size of Change Agent grants 
and on the number of students served by the projects, we can derive 
a measure of the ooncentration of innovation resources. The project 
grant per pupil in the target group (in the project's biggest fund- 
ing year) can be taken to indicate several phenomena: 

o the expense per pupil of introducing and maintaining 
a project , 

o the degree of focus on particular students (or dis- 
persion among students) in a project, or 

o the policy intentions and priorities of the funding 
source. 

Table 8 shows the pattern of concentrating resources for each of the 
federal grant-in-aid programs. Only the state Title III program has 
enough observations to permit good comparisons, but we may note that 
it supports more projects wKich spend less than $25 per student 
(24.8 percent of State Title III projects than do the other federal 
programs (11.9 percent of the five other program projects). Still, 
almost a fifth of the projects spend more than $450 per target 
student. There is clearly variation in the concentration of 
resources for innovation; like the other individual-project- 
history items, this variable may or may not have a real effect on 
project outcomes. Still, these measures are so commonly used that 
they give us a sense of what the projects, taken together, are like. 

To further clarify the relationships between the concentration 
of resources, federal programs, and types of projects, we examined 
the level of federal funding received by a project as a function of 
the size of the number of students being served by the project and 
the type of project it was. Tables 9A-D present relevant regressions. 
For Right-to-Read , the more students there are, the higher the fund- 
ing as is to be expected from the funding formula for Right-to-Read; 
for Bilingual the effect is also present. However, for the Title III 
programs, the size of the grant depends only slightly on target 
group size even when the type of project is controlled. 

In addition to the question of funding from the federal program 
of concern to the Rand study, an analysis parallel to the one 
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sketched above was conducted for the extra funds received by a project 
from other federal programs, the state, local sources, or foundations. 
Table 10 A-C presents the results. 
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INSTITUTIONAL SETTING 



Many elements of Institutional setting will be considered In 
the analysis. For example, teacher, principal, and project director 
personal characteristics and such school characteristics as size, 
percentage of Title I students, and morale will be Included In the 
analysis of teacher implementation outcomes. At the level of the 
school district, we wish to consider various LEA characteristics 
as a complex whole rather than Isolated facts. This section deals 
only with the school district level and offers a preliminary 
analysis of a single measure that suniaarlzes a number of critical 
LEA characteristics. 

One of the strongest impressions gleaned from our field expe- 
rience was the importance of the overall school district to in- 
novativeness. That is, some school districts seemed more likely 
to innovate and seemed more likely to produce successful in- 
novations than other districts. The literature on educational 
Innovation often ignores the Institutional setting and those 
studies that do analyze organizational aspects usually focus on 
the school but neglect the district. We believe the school dis- 
trict does matter; the challenge is to understand how. 

The strategy for dealing with school district effects involves 
two stages. The first stage consists of developing an equation that 
predicts the propensity to Innovate of a school district based upon 
various structural characteristics of the district. The second stage 
consists of using both the predicted value of Innovativeness and 
the residual value of innovativeness (i*e,, the extent to which 
districts "over" or "under" innovate according to their structural 
characteristics) in the analysis of the iucre«B of individual 
projects. The rationale for this two-stage strategy rests on our 
hypothesis that the structure of the school district affects 
schools, teachers, and projects as a complex whole; we siimmarize 
this by the district's propensity to innovate. The second stage will 
be discussed in detail In the final report. This section presents 
a preliminary analysis of the first stage. 
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MEASURING INNOVATIVENESS 

Innovativeness is an elusive concept. Not only is there no 
agreed upon definition of innovativeness in the literature but 
operational measurements differ widely. We view innovation as a 
change process involving various stages. Different conceptions of 
innovation emphasize different stages in the change process. One 
conception deals with the invention of new techniques, strategies, 
or arrangements. The invention of educational strategies, methods, 
or technology is not the focus of this study. Rather we are con- 
cerned with the change processes initiated by the adoption of 
projects or programs that are new relative to the adopting school 
district Cor units within the sciiool district). Alternative 
definitions of innovation focus on two other aspects of change 
processes. First, the successful implementation of a project or 
program that is new relative to the district. Second, innovation 
may also be taken to mean that the introduction of a project or 
program produces a presumably improved outoonts. Implementation 
and outcomes will be treated in Volumes II and III but, for 
reasons to be elaborated, the adoption definition is the one 
employed in this section. 

School districts differ widely in their propensity to (and 
the rate at which they) adopt projects or programs new relative 
to their current practices. Without begging the question of the 
extent to which the adoption of a new program implies either its 
full implementation or significant outcomes, we shall operationally 
measure the propensity of school districts to adopt "innova- 
tions" by summing up the number of widely discussed educational 
innovations tried by the district in the last decade. 

Table 11 presents a list of 21 "educational innovations."* Each 
superintendent of the school districts in our sample (n « 194) was 



^Superintendents were also asked about the adoption of bilingual 
programs. However, since such programs are only adopted in LEA^s 
having significant non-English speaking pupils, they are dissimilar 
from other educational innovations on the list used, hence, were 
deleted. 
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Table 11 

Superintendent's Responses to List of Innovations in the 

School District 



Question: Here is a list of eduoational innovations which haoe been tried 
out in some school districts in the last decade. For each innovation, please 
circle the appropriate code to indicate whether it was never tried, tried 
but not incorporated, or has been incorporated into current practice in your 
district. 



Tried in 
District But 



Incorporated 
into Current 





Never 
Tried 


Not Now 
Incorporated 


District 
Practice 




Programmed learning 


19.21 


28.25 


52.54 


NA-17* 


Extended school year 


72.88 


3.38 


23.72 


NA-17 


Extended field trips 


17.77 


10.55 


71.66 


NA-14 


Team teaching 


5.11 


5.68 


89.20 


NA-18 


Non-graded or ungraded classrooms 


15.08 


10.05 


74.86 


NA-15 


Flexible scheduling 


20.00 


15.42 


64.57 


NA-19 


PSSC (Physical Sciences Study Committee) 
Physics 29.82 


14.03 


56.14 


NA-23 


Typing in elementary 


57.71 


18.85 


23.42 


NA-19 


Community school 


54.71 


7.65 


37.65 


NA-24 


Work/ Study program 


9.09 


3.41 


87.50 


NA-18 


Teacher Corps 


71.67 


8.67 


19.65 


NA-21 


Student exchange 


30.81 


18.60 


50.58 


NA-22 


Educational T.V. 


13.55 


15.25 


71.18 


NA-17 


Simulation or gaming 


26.90 


19.29 


53.80 


NA-23 


Individualized instruction (Method 
and/or Materials) 


.56 


5.08 


94.35 


NA-17 


Open classrooms 


23.03 


8.43 


68.54 


NA-16 


Program budgeting (PPBS [Planning 
program and budgeting system]) 


52.80 


13.48 


33.70 


NA-16 


Behavioral objectives 


7.91 


14.12 


77.97 


NA-17 


Alternative school 


46.86 


7.43 


45.71 


NA-19 


Special classes for gifted 


22.15 


17.04 


60.79 


NA-18 


Needs assessaent 


10.11 


15.17 


74.72 


NA-i6f 



NA - nuaber of superintendents not answering. 
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asked to indicate for each innovation whether the school district 
had tried the new program and whether the program was currently 
incorporated into district practice. Some of these educational 
innovations — e.g., extended school year — if fully implemented 
would require extensive changes in the administrative life of a school 
district; others, such as the introduction of educational television, 
imply fewer organizational changes . The frequencies of responses 
shown by Table 11 indicates that such practices as team teaching, 
work/study program, and individualized instruction have been gener- 
ally adopted by the districts in this sample whereas other practices 
such as extended school year, teacher corps, and program budgeting 
(PPBS) have not been as widely adopted. The question this section 
asks is what aspects of the characteristics of school districts (and 
how they vary) explain the differences in their propensity to adopt 
innovations of the type indicated in the above list. 

The analysis could proceed by considering each innovation sep- 
arately as a function of theoretically plausible characteristics of 
the district and then comparing the results for each innovation • 
Since that procedure would be costly and might tend to be dominated 
by the specific substance of the list of educational innovations 
used here, we approach the measurement of the dependent variable by 
aggregating the individual measures. In particular, we employed the 
following five scales: 

(1) An unweighted sum of the responses to all the educational 
innovations where a zero score was given if the district had 
not tried the innovation and a score of one otherwise. 

(2) An unweighted stm of the responses to those innovations that 
primarily involve the student in this classroom and do not 



The responses to whether the programs were "tried in the dis- 
trict but not now incorporated" are ambiguous because a yes response 
could indicate either that the project was tried and rejected or is 
being tried and has not yet been incorporated* To avoid errors due 
to this ambiguity the analysis categorizes the responses for each 
innovation into either never tried or tried. 
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imply major administrative changes in the school or school 
district organization, 

(3) An unweighted sum of the responses to those innovations that 
imply administrative or organizational changes in the school 
or school district. 

(4) A weighted sum of the responses to those innovations that 
imply administrative or organizational changes in the school 
or school district. 

(5) A weighted sum of the responses to all innovations using the 
weights of scale four. 

The point of using these alternative s ceiling procedures is to 
explore several problems of validity in the construction of an Innovation 
index. First of all, the summing of the various items tends to mask 
overly strong effects of the specific substance of each item. However, 
such aggregation necessarily makes the index abstract and thus should 
be interpreted as the propensity to adopt current educational innova- 
tions. Secondly, since the aggregation of all items might lose "too 
much" of the substance of the innovations, scales two and three sep- 
arate the student-class oriented innovations not implying administative 
changes from those innovations that involve administrative alterations. 
Thirdly, the various innovations undoubtedly differ in the ease with 
which they might be adopted; an equal weighting scheme assumes away 
these differences. Scales four and five represent a preliminary 
effort to weight the innovations and thus enable us to examine the 



The items included are programmed learning, extended field trips, 
PSSC physics, student exchange, educational TV, simulouion or gaming, 
individualized instruction, and special classes for gifted. 

The items included are extended school year, team teaching, non- 
graded or ungraded classrooms, flexible scheduling, community school, 
work/sttdy program, teacher corps, open classrooms, PPBS, behavioral 
objectives, alternative schools, and needs assessment. 

The following weights were used: three for alternative school; 
two for each of open classroom, non-graded or ungraded classrooms, and 
team teaching; and one for the remaining administrative items. 
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sensitivity of the results to an equal weighting assumtion. Table 
12 presents the statistical characteristics of the five innovative- 
ness scales* 

EXPLAINING SCHOOL DISTRICT INNOVATIVENESS 

In "explaining" innovativeness in this section, we will not 
deal directly with internal processes or decisions within the school 
districts. Rather we will try to explain the differences in the pro- 
pensity to adopt innovations in terms of theoretically plausible 
characteristics of the district. Considerable empirical litera- 
ture about the diffusion of innovation, particularly in the fields 
of agriculture, medicine, public bureaucracies, and economic firms 
suggests that size, wealth, and the availability of resources are 
related to the propensity of organizations to adopt innovations,* 
That is , larger and wealthier organizations often appear to adopt 
more innovations. This finding may seem contrary to one's intuition 
about the "conservative" nature of large organizations and perhaps 
even more counter-intuitive for school districts. Nonetheless, as 
the following analysis shows factors related to size and wealth, 
strongly affect school district innovativeness. 

Table 13 presents the results of a statistical analysis of the 
school district's propensity to adopt innovations considered as 
determined by a variety of characteristics of the district. In par- 
ticular, using ordinary least squares estimation procedures, the 
independent variables used to explain the variation in school district 
innovativeness represent five groups of factors that, on a priori 
grounds, might affect innovativeness. The first group consists of 
two measures of size — total LEA enrollment and number of students per 
school; the second group consists of four measures of the district's finan- 
cial situation—the expenditure per pupil of the district measured 
in terms of the district's deviation from the state's average, the 
average expenditure per pupil of school districts in the district's 
state, the general financial situation of the district as assessed 
by the superintendent, and whether the district (according to the 
superintendent) has been forced to cut back on programs due to 



*Thi» prellfldnary Vtorklng Note generally omits specific references to 
O the literature. 
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Table 12 

STATISTICAL CHARACTERISTICS OF INNOVATIVENESS SCALES 



ERIC 



Scale 



Mean 



Sum of All Innovations Unwt. 13.9 

Sum of Student-Class Innov. 6.8 

Sum of Administrative Innov. 7.1 

Sum of Administrative Innov. 

Wt. 9.8 



Sum of All Innovations Wt. 



16.6 



Median 

13.9 
7.1 
7.2 

9.9 
16.8 



St. Dev. 

3.5 
1.8 
2.2 

3.2 
4.5 



Range 

0-20 

0-9 

0-11 

0-15 
0-24 



SCHOOL DISTRICT IMMOVATIVENESS 



R£6RESSI0M COEFFICIENTS FOR 

imiOVATIOM INDICES 
(Std. trror) (Frob-Valu«) 



School District 
Independent Variables 



All 



CU«» 



Mmin. Ut. 



All Wf. 



(Log) District SnrollBent 



J Pupils Per School (Residual) 



Expsndlturss Per Pupil/State 
Average 



Stata Average Expenditure 
Par PupU 



Ada<{uacy of District Financial 
Situation 

Raccac Cutbacks in Prograas 



Parcant Ravanue fro* State/ 
Stata Average 



Stata Average Percent Revenue 
Proa Stata 



Parcant Ravaoua from Fedaral/ 
Stata Average 



Stata Average Percent Revenue 
from Federal 



Parcant Faailias vlth IncoM 
125,000 



Parcant Poor FsMlliaa Predicted 
from Mlaority 



Rural 



Taauro of Supartataodaat 



.Tears of SopariAtaadaat'e Prior 
la District 



1.49 
(.168) (.00) 

* 

-X.85 
(.518) (.00) 



.183 
(.667) (.78) 



-.0005 
(.001) (.73) 

** 

.396 
(.240) (.10) 

-.890* 
(.416) (.03) 



-120. 
(48.3) (.01) 



.036 
(.021) (.09) 



6.60 
(16.8) (.70) 

-.104 
(.969) (.29) 



12.7 
(5.55) (.02) 



-U.3 
(5.U) (.03) 

-.858 

(.687) (.21) 

.131* 
(.038) (.00) 



.031 
(.024) (.U) 



.697 
(.101) (.00) 

-1.03* 
(.312) (.00) 



-.073 
(.402) (.86) 



-.0003 
(.001) (.71) 

.244** 
(.144) (.09) 

-.250 
(.251) (.32) 

-56.3** 
(29.1) (.06) 



.005 
(.OU) (.67) 



3.96 
(10.1) (.70) 



-.015 
(.058) (.80) 



6.93 
(3.35) (.04) 



-8.35 
(3.08) (.01) 

-.159 
(.414) (.70) 

.070* 
(.023) (.00) 



•020 
(.015) (.18) 



.790 
(.113) (.00) 
* 

—.827 
(.348) (.02) 



.256 
(.448) (.57) 



-.0007 
(.001) (.40) 

.152 
(.161) (.35) 

* 

-.640 
(.279) (.02) 



-64.9 
(32.4) (.05) 



.030 
(.014) (.03) 



2.64 
(U.3) (.81) 



—.089 
(.065) (.18) 



5.78 
(3.73) (.12) 



-2.91 
(3.43) (.39) 

-.698 
(.461) (.13) 

* 

062 
(.026) (.02) 



.OU 

(.016) (.51) 



1.10 
(.182) (.00) 

-1.19* 
(.561) (.04) 



•664 
(.722) (.36) 

-.0004 
(.001) (.78) 

.345 
(.260) (.187) 

^ ** 
-.827 

(•450) (^07) 



-65.8 
(52^2) (.21) 



•030 
(.023) (.19) 



16.1 
(18.2) (.38) 



-•057 
(•105) (^59) 



U^3 
(6^01) (.06) 



-5.30 
(5^54) (^34) 

-1^31 
(•743) (^08) 

.073** 
(.041) (.08) 



.019 
(.026) (.48) 



1.80 
(.226) (.00) 

* 

-2.22 
(.697) (.00) 



•592 
(.900) (.51) 



-.0001 
(.002) (.95) 

.588** 

(.323) (.07) 

-1.08** 
(.56) (.06) 



-122. 
(65.0) (.06) 



• 036 
(.028) (.21) 



20.0 
(22.6) (.38) 



-.072 
(.130) (.58) 



18.2 

(7.47) (.02) 



-13.6 
(6.88) (.05) 

-1.47 
(.ft4) (.U) 

•U3* 

(.51) (.01) 



•OM 

(.033) (.24) 



R^ (Corractad) 
Ootraiatiott Coaf f iciant 
Dagraas of Fraadosi 

#f iTipMimr larlaiaA 



.58 
•79 
134 
0-10 



.44 
.70 
134 
0-9 



.49 
•74 
134 
0-U 



.41 
•69 

134 
0*15 



•54 

•7i 

134 



:erlc= 
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financial circumstances; the third group relates to the source of the 
district's revenues — the percent of the district's revenue derived 
from local sources measured in terms of the district's deviation from 
the state aver age, and the average percentage of revenue derived from 
the local sources of school district in the district's state; the 
fourth group consists of three measures of the socio-economic-ethnic 
characteristics of the community in which the school district is 
embedded — the percentage of families in the community with incomes 
over $25,000, the percentage of families in the community with incomes 
under the poverty level combined with the percentage of families from 
minority groups (black or Spanish speaking), and whether the district 
is in a rural area; and the final variable is the tenure of the dis- 
trict's current superintendent. In the course of analyzing the 
results, we will discuss the meaning of these variables more fully 
and interpret their theoretical significance. 

Table 13 displays the regression coefficients from each of the 

independent variables along with their standard errors and probability 

values. The same structural equation is used for the five measures 

2 

of propensity to adopt innovations previously discussed. R , the 
proportion of the variation (adjiisted for the degrees of freedom used 
in the estimation) explained by the independent variables is indi- 
cated below each column. 

The variable with the largest effect on the propensity to adopt 
innovations is the size of a school district as measured by enroll- 
ment: Controlling for other factors, it accoimts for approximately 
30 percent of the variance in the dependent variables. Why do larger 
school districts tend to adopt more innovations? One theoretically 
plausible explanation is that larger school districts have more 

Since the distribution of enrollment is highly skewed, a loga- 
rithmic transformation of the enrollment was employed leading to a 
much closer to normal distribution. The regression coefficient for 
log enrollment should be interpreted as ther^ being an average in- 
crease of 1.56 innovations for every change of one in the natural 
logarithm. 
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"organizational slack" than smaller districts. Organizational 
slack in the context c£ the school district can occur in 
several ways» Larger school districts have, larger operating 
budgets and perhaps a greater flexibility to direct funds within 
that budget. Perhaps even more importantly, size may allow polit- 
ical flexibility. That is, the motivation for school districts to 
adopt innovations are complex and mixed. For any of a variety of 
specific reasons , the district may feel either a positive desire or 
a reactive need to introduce new programs. On the other hand, dis- 
trict decision-makers may be risk adverse in the sense that they may 
be more concerned with avoiding failures than promoting change • All 
other things being equal, the consequences of failure of a project 
with the same scope in a large district would have fewer and more 
diffuse political repercussions than in a small district. In short, 
bigger districts can better afford, in political terms, to experiment 
than smaller districts. 

Several implications follow from this size effect for the pros- 
pects of particular innovations. For a project of the same relative 
"scope," we would expect there to be less pressure from above in a 
larger district than in a smaller one. We do not have plausible 
theoretical reasons to suggest what the direction of the effect of 
such pressure on the success of a project might be; we take this 
question to be an empirical issue. However, it is reasonable to 
hypothesize that assuming equal success of an experimental project 
of the same relative scope, the larger district would be less likely 
to incorporate a project — i»e., propogate it throughout the district 
on a regularized basis. For the smaller district, the lack of polit- 
ical slack means that adoption is equivalent to the placing of one's bets 
For the larger district, the availability of political slack means the 
adoption is experimenting; when the time comes to incorporate, the 
pressure is to spread the innovation throughout the district and 
such propagation raises severe political risks ♦ 

We have deliberately glossed over the phrase "relative scope" 
of a project. It is clear that a project involving the same absolute 
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level of resources (teachers , materials , and expenditures) has 
different economic consequences in a small district than in 
a larger district* In particular, the same project in the 
small district can be expected to have a larger opportunity cost than 
in a large district. Such opportunity costs cannot be calculated but, 
nonetheless, play an important role in district decisions. Moreover, 
identical projects may have more political visibility in a small 
district than in a large district. 

Another dimension of district size also affects innovation. Dis- 
tricts having the same overall enrollment differ in the number and 
size of the schools within the district. To capture part of this 
important organizational difference among school districts, we used 
as an explanatory variable the enrollment density for the district— 
i.e., the average number of students per school in the district. 
Density decreases innovation, all other things being equal. Though 
density's impact on innovation is about one third as great as that of 
enrollment, it is highly significant as Table 13 shows. 

Several plausible explanations of the importance of density can 
be offered. Perhaps the most compelling theoretical reasons are based 
upon organizational slack. The more dense Uie district (and thus the 
fewer the number of schools for the same enrollment), the less slack 
exists both in economic and political terms. 

In addition to size related characteristics, the effects of 
'^wealth" (controlling for size and other wealth-related characteris- 
tics) can be expected to affect the propensity to adopt innovations. 
Measuring the wealth of a school district is an extraordinarily com- 
plex task. Not only is it difficult to conceptualize. wUat the 
appropriate measures of wealth should be, but gathering the ap- 
propriate information from school officials, who cannot be 
expected to keep their financial records in a theoretically 

H 

The enrollment density increases with larger enrollment. (Over- 
all [log] enrollment is correlated .485 with enrollment per school.) 
iSince we are interested in the effect of density independent of enroll- 
ment, the variable used in the regression is the residual of [log] 
pupils per school regressed on [log] enrollment. 
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useful way is, at best, uncertain. As a means for coping with this 
situation, we used four measures. Expenditure per pupil varies con- 
siderably across individual school districts throughout the country. 
However, part of this variation is due to differences among the 
states in such areas as state policies and regional wage rates. 
Since our sample of school districts was picked by a first-stage 
state selection, we need to control the effect of state on the ex- 
penditures of school districts within the state. Therefore, the 
regression includes both deviation of each school from the state 
average expenditure per pupil and the absolute value of the state 
average. Neither of these variables were significantly related to 
innovativenesc . 

That the state average expenditure per pupil fails to be sig- 
nificant is not surprising. Much of this difference in averages 
among states is due to such costs as teacher salaries and operating 
and maintenance costs of districts rather than to the type of addi- 
tional expenditures that might result in economic slack.* 

The lack of significance of the relative expenditure per pupil 
is more surprising—at least at first thought • However, many of the 
reasons school districts sperd more money per pupil than other dis- 
tricts (in the same state) arise from dealing with problems such as 
compensating for children from poor or minority families • Such 
incremental expenditures mitigate against slack for the district. 
On the other hand, higher relative expenditure per pupil can reflect 
a greater local wealth base of the district and political pressures 
from the wealthier members of the community; in this case, one does 
expect slack and an impetus towards innovation* In short, the dif- 
ferences in relative expenditure per pupil arise from conflicting 
sources and thus cause expenditure per pupil not to be significantly 



For example, the range of the average of the salaries of the 
instructional staff for the highest "-paying state (New York) in our 
sample to the lowest-paying state (Arkansas) in our sample is 
$11,730 to $6,715. [Ranking of the States, 1972*1 
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related to Innovativeness when those other sources are taken into 
account . 

Whereas expenditure per pupil has little effect, two other direct 
but "noisy" measures of the financial situation do affect the propen- 
sity to adopt innovations. We asked each superintendent to indicate 
what the present financial situation in the district was and to in- 

icic 

dicate whether the district had been forced to cut back on programs. 
Since both of these variables rely on the subjective judgment of supei-- 
intendents, their validity needs to be questioned. Nonetheless, it 
seems reasonable to interpret the superintendent's answer to the 
financial situation for carrying out needed educational programs as 
a surrogate for the extent of economic slack (excesi> over needed 
funds) available in the district. In any event, the better the fi- 
nancial situation, the higher the propensity of districts to innovate. 
Half the superintendents in our sample said they had to cut back on 
programs due to financial shortages. The districts saying so are 

The data support the above explanation for the lack of signif- 
icance of the relative expenditure per pupil in the following way. 
The zero-order correlation between relative expenditure and the inno- 
vativeness scale is approximately .2 but expenditure per pupil also 
has a zero-order correlation of .13 with the percent of the families 
in the district who are black or Spanish-speaking and .19 and .12 
with the percent of the families with incomes over $25,000 and the 
overall financial situation of the district as reported by the super- 
intendent respectively. The analysis includes and controls for all 
of these variables (as well as the others indicated in Table 13) in 
which event expenditure has a positive but not significant effect 
upon innovativeness with a partial correlation under .1. 

ick 

The questions and the marginal results are. the following: 

How do you view the present financial situation in your district? 
Would you say your budget is more than adequate^ adequate^ barely ade^ 
quote ^ or inadequate to carry out needed educational programs? 

% 

More than adequate 29.8 

Adequate 26.7 

Barely adequate S9.8 

Inadequate 3.7 

Ho answers -S, n ~ 194. 
Has your district been forced to cut back on programs in the last 
few years as a result of financial shortages? 

Yes 49.0 

No 51.0 

No answers -2^ n » 194. 
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districts on the financial margin and would be less likely to have 
economic slack; they were less likely to adopt innovations. 

The above findings can be further clarified by examining the 
analysis of class-type innovations versus administrative innovations. 
Table 13 shows that for the subset of class-type innovations the 
general financial situation is significant (indeed more so than for 
the innovativeness scale including all innovations) whereas cutback 
fails to be significant. The opposite result holds for administra- 
tive-type innovations. A plausible interpretation of these results 
is that districts having economic slack are more likely to adopt 
class^student type innovations and districts operating at the finan- 
cial margins are less likely to adopt administrative-type innova- 
tions . 

One important implication of the above findings for particular 
innovative projects is clear: all other things being equal, we would 
expect districts having financial slack to be more likely to continue 
a class-student type project on its own funds after initial federal 
funding is completed than an administrative -type project. 

The next group of variables is concerned with the source of 
financial support for school districts. There is considerable vari- 
ability among school districts in the extent to which their revenues 
come from local government, the state, or from the federal government. 
A maior aspect of this variability is related to the differences among 
states in terms of their wealth, their demographic characteristics, 
and their state policies towards the financing of education. For 
example, the percent of revenue for schools from state government 
varies in our sample from a high of 68.7 (North Carolina) to a low 
of 21.7 (Massachusetts). To correct for state variation, the analysis 
uses both the absolute value of the state average percentage and the. 
relative deviation of the school district from its state average* 

The results of the analysis shown by Table 13 indicates that dif- 
ferences among school districts in the percentage of federal funds 
they receive vis & vis revenue from state and local governments does 
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not significantly affect the propensity to adopt innovations. This 
finding is xmderstandable when one considers the "entitlement" or 
grants-in-aid basis for federal funding. Of course , this result 
does not imply that a particular innovative project funded by the 
federal programs considered in this study would not work. 

The differences among school districts in the percent of their 
revenue derived from state government does significantly affect in- 
novativeness. The higher the percentage of revenxie from the state 
relative to other school districts within the state, the lower the 
propensity to innovate. Two hypotheses may explain this result. 
First, incrementally more state money may be going to districts that 
need the funds in order to deal with their financial problems. Thus, 
the relative deviation of a district from the state average in the 
percentage of funds received from the state is negatively correlated 
with both the relative expenditure per pupil (-.31) and the general 
financial situation of the district (-.24); it is also negatively 
related (-.31) to a measure that in part reflects the wealth base of 
the community, the percentage of families with income over $25,000. 
Moreover, the relative percentage revenue received from the state 
is highly negatively correlated (-.89) with the relative percentage 
revenue received from the local government. Secondly, state monies 
may be more tied down than either local or federal funds. Both of these 
hypotheses work in the same direction towards reducing the amount of 
slack for the school district. 

The next group of variables, representing demographic character^ 
istics of the community, are importantly related to innovativeness. 
The percentage of families with income of at least $25,000 increases 
innovativeness whereas the percentage of families with income below 
the poverty level combined with minority decreases innovativeness. 
Rural decreases innovativeness as anticipated, though the result 
fails to be significant. 

The final group of variables refer to a factor that analysts of 
school systems believe strongly influences the policies of the school 
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district—the superintendent. The tenure of a st5>erintendent 
(whether measured in terms of numbers of years as superin- 
tendent or as a dtimmy variable for three or fewer years or more than 
three years) increases innovativeness • This result holds up even 
controlling for the mobility and the past experience of the superin- 
tendent. An hypothesis explaining this result is a political- 
organizational one. Innovations in school districts generally come 
incrementally and involve the ability of the superintendent to use 
the political slack in the system. More experienced superintendents 
better know how to manipulate their system. 

Further interpretations of these preliminaiTr findings will 
be deferred. Volume II of the Change Agent reports will treat 
school district innovativeness more extensively and will integrate 
the findings presented by this Working Note into the more com- 
prehensive analysis of the school district's decision to continue 
innovative projects. 




i 



APPENDIX 




58 



Appendix 

REPRESENTATIVENESS OF SURVEY AND FIELD WORK SAMPLES 

I , INTRODUCTION 

The federal government sponsors education Change Agent programs 
to introduce or spread innovative practices at the local school dis- 
trict level. Rand is attempting to identify for federal, state, and 
local policy-makers what characteristics of the programs themselves, 
the innovations they support, or the districts that adopt them lead 
to successful implementation and continuation. A series of hypotheses 
concerning innovation in public schools will be tested with data from 
a nation-vide survey of Change Agent projects conducted by the Na- 
tional Opinion Research Center of Chicago (under subcontract with 

it 

Rand). This survey involves 187 school districts, and is supple- 
mented by in-depth field studies in 23 school districts . 

This paper uses census variables to characterize the survey and 
field work samples being used in the Change Agent study. To test 
the representativeness of the samples , it compares the survey sample 
with the total population, and compares the field work sample with 
the survey sample. In addition, the Appendix briefly describes each 
of the field sites. 

The sampling of Change Agent projects was accomplished in two 
stages: a sample of 18 states from the contiguous 48 followed by 
the selection of projects within states. The general objective of 
the state sample was to obtain a sample broadly representative on 
three dimensions: region, level of education funding, and intensity 
of educational management at the state level. For the selection of 
projects, the guiding objective was diversity of school districts — 
i.e., large and small districts, ur.ban and rural, varying racial- 
ethnic concentrations . 



There were 18,655 operating school districts in the U.S. in 
fall 1969 [41. 



59 



Several criteria were used to determine eligibility of projects 
for the sample. The one most likely to affect the representativeness 
of the sauple may be the requirement that the project have a yearly 
funding level of at least $10,000, which would bias the sample to 
some degree toward larger districts . Because of the weighting cri- 
teria used in the selection of the 18 states, 6 states fell into the 
sample with certainty. These 6 are California, Illinois, New York, 
Ohio, Pennsylvania, and Texas, which as a group contain most of the 
largest metropolitan areas . So , we expect our survey sample to be 
biased to some degree toward larger school^p!«tricts . More details 
on sampling procedure can be found in [2; pp. 30-42, 74-76]. 

For the field work sample, five field work teams, each focusing 
on one area of innovation, selected projects from 23 school districts 
for in-depth study in the areas of career education, bilingual educa- 
tion, classroom organization, staff development, and reading. The 
districts chosen for each of the five areas vary over organizational, 
financial, and political conditions, allowing observation of the 
effects of such variables on the particular innovation. 

We discuss the census variables for the survey sample and test 
its representativeness in Section II. The field work sample is given 
a similar treatment in Section III. 



II. THE SURVEY SAMPLE 



Census Characteristics 

We are using as our data source the 1970 Census School District 
Data Tape which contains the usiial census-type variables organized by 
school districts. We will use six variables to describe the sample. 
These are total population of the district, public school enrollment, 
proportion of the population in the district that is urban, propor- 
tion that is black, and two income variables: the proportion of 
families with incomes in 1969 of $25,000 or more and the proportion 
of families below the poverty level in 1969. 

From a priori considerations, we use logarithms of both total 
population and school enrollment. Transformations for some of the 
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other variables may be suggested by the statistics for the survey 
sample. Such transformations may be useful In the modeling of Inno- 
vatlveness of school districts. 

All of the data analysis In this paper will treat New York City 
separately. All of New York City Is one school district, and statis- 
tics on the data tape are all aggregated to the whole city. However, 
the city is divided into what are called community school districts, 
which operate with a certain degree of autonomy. Our survey of 
Change Agent projects Includes 9 of the 32 community school districts, 
and data for these 9 community districts are not on the census tape. 
Data for some of the six variables we are using are available from 
the New York City Board of Education, and will be discussed later. 
But in the discussion immediately following, we are excluding New York 
City. Our sample, then, consists of 186 school districts. 

Transformed (to natural logarithms) values will be used for 
population and enrollment, and untransf ormed proportions for the other 
four variables. The means for the logs of population and enrollment 
are 10.580 and 8.990 respectively, corresponding to population and 
enrollment values (i.e., exponentiating the mean logs) of about 
39,000 and 8,000 respectively. Log enrollment ranges from 5.790 to 
13.394, corresponding to an enrollment range of 327 to about 650,000. 
The log transformations for both population and enrollment produce 
variables that are very symmetrical and nearly normal in distribution, 
as indicated by the near equality of mean and median, and by the 
measures of skewness and kurtosls. More summary statistics for these 
two variables are given In Table 1, and histograms are In Tables 2 
and 3. 

The proportion of the population of a school district that is 
urban covers the complete range from 0 to 1 in our survey sample of 
school districts, with a mean proportion urban of .740. The median 
proportion urban, however, is .931, Implying some skewness to the 
left in the distribution. As seen in the histogram In Table 4, 
just over 40 percent of the districts are between 98 and 100 percent 
urbaa. Still, about 15 percent of the districts are below 20 percent 
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urban. Additional summary statistics for proportion urban are in 
Table 1, 

The statistical behavior for proportion black in the 186 survey 
sample school districts is similar to that for proportion urban, on 
the opposite end of the range. Just over 40 percent of the districts 
have less than 2 percent black. The mean and median proportions 
black are respectively ,105 and ,052, with a range of 0 to ,583 
(these are proportions black in the total population of the school 
districts, not black enrollments in the districts or individual 
schools). Additional summary statistics and a histogram are in 
Tables 5 and 6 respectively. 

Finally, we consider the two income variables: proportion of 
families over $25,000 income and proportion of families below the 
poverty level. Their means are respectively ,046 and ,107, They 
are both somewhat skewed to the right, with a few extreme values. 
Proportion over $25,000 has a maximum of ,359, but only 8 percent 
of the districts are over ,100, Similarly, the proportion of fam- 
ilies in poverty is as high as ,469, but only 10 percent of the 
districts are over ,200. See Tables 5, 7, and 8, Both income 
variables are rather sharply peaked, especially the proportion over 
$25,000, The proportion over $25,000 is between ,02 and ,05 for 
half of the districts. 

For an indication of relations among these six variables in 
the sample, see Table 9, 

The shape of the distribution for proportion of families over 
$25,000 suggests a possible transformation for use in the modeling 
of innovativeness, A square root transformation, e,g,, sharply 
reduces the kurtosis , It also brings the mean and median much 
closer together. Square root transformations on proportion poor, 
proportion black, and proportion urban have similar effects, though 
not nearly so dramatically as with proportion above $25,000, 

We now want to consider how these sample statistics compare 
with national averages for these variables. 
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Repr es en t a t iv enes s 

We would like to know the distributions of the six variables we 
are using over all school districts in the U.S. Testing the repre- 
sentativeness of the sample would then be a straightforward matter. 

What we have instead are estimates of the national means for the 
six variables, gathered from a variety of sources. They are national 
aggregate figures, not means over school districts. And we must 
rely on the sample itself for variance-covariance est5.mates. From 
[3], we have national means for proportion of population that is 
urban in 1970 of .735, and for proportion black .111. From [5], we 
have estimates for proportion of families over $25,000 income in 
1969 of .063, and for proportion poor .097. From [3 and 4], we have 
total population in 1970, total public school enrollment in fall 
1969, and number of operating public school districts, from which we 
calculate log of population per district and log of enrollment per 
district as 9.926 and 7.777 respectively. We thus have significant 
overestimates of population, enrollment, and proportion poor, and 
especially for the first two. We have an imderestimate for propor- 
tion of families above $25,000 income. Proportions urban and black 
are almost equal to the population estimates. 

2 

Using a multivariate generalization (the T -statistic) of the 
t-test, we test for equality of vector means between the sample and 
population. We find that the difference between the two vector means 
is statistically significant at the .001 level , implying that the 
survey sample is not very representative of the total population. 



For p = the vector of population means, x « the vector of 
sample means, S « sample covariance matrix, N. ■ the number of 

observations (in this case 186), and p * the number of variables 

2 2 ^ 

(in this case six), the T -statistic is given by T - N(x-V4)' 

S-^x-y). Then . (N-p) ^^^^.^M , See. e.g.. Anderson [1]. 
.(N-Dp ' 

In this case. T - 279.66. T . (N-p) 45.35. and the .999 point 

(N-l)p 

°^ ^6.120 - ^-O^- 
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It appears that the survey sample has significantly larger 
than average school districts. This is at least partly because of 
selection criteria that eliminated from consideration the very small 
school districts (see Section I, and [2]), The smallest in the 
sample has an enrollment of 327, In fact, more than one-third 
(36.9 percent) of all public school districts in fall 1969 had fewer 
than 300 students enrolled. But this accounts for only 1,5 percent 
of all students enrolled, so log of enrollment per district would 
increase only about ,44 if we calculated the population mean only 
for districts with enrollments of more than 300 (the enrollment 
figures are from Table 1 of [4] • The difference between population 
and sample means would still be more chan 7 times the standard error 
of the estimate of the sample mean. In fact, even if we eliminate 
both population and enrollment, and run the test on just the other 
four variables, the difference between population and sample vector 
means is still significant at the .001 level. 

We should, of course, recall that our population means are not 
really means over all school districts; they are means calculated 
from aggregate data* To the degree that the means used here reflect 
the true population means over school districts, our survey sample 
is not very representative of school districts in general. Our 
districts are much larger both in population and school enrollment, 
have proportionately more poor families and fewer families who had 
incomes over $25,000, The sample seems representative on the whole 
in terms of proportion black and proportion urban, though these 
variables are both highly skewed in the sample. To the degree that 
our population means are accurate reflections of school district 
means, we can clearly say that the school districts in our survey 
sample are not typical school districts, at least in terms of the 
variables considered here. This limits the generalizability of the 
results . 



The formulation is the same as before, except that now p « 4, 

— 2 
and y, x and S are reduced in size« For this case, T « 33.18, 

T^(N-p) ■ 8.16^ and the .999 point of F, « 4.95. 
(N-l)p ^•"^ 
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What Is suggested by the above discussion Is that school dis- 
tricts that have any Change Agent projects at all, regardless of how 
successful they may be, are atypical school districts. Our results 
may then be generalizable to all school districts that have at least 
one project in operation* 

New York City 

As pointed out earlier. New York City is not included in any of 
the above analysis. Nine of the 32 community school districts within 
New York City are included in the survey sample. These districts are 
compared to city aggregates in Table 10. The nine community districts 
included in the survey sample appear to be representative of city 
aggregates, at least on the three variables population of school 
district, public school enrollment of the district, and proportion 
of population in the district that is black. 

III. THE FIELD WORK SAMPLE 

Census Characteristics 

To describe the field work sample, we use the six variables used 
in Section II, with the addition of proportion Spanish language, i.e., 
the proportion of the population in the school district for whom 
Spanish is the major language. Again, New York City will be con- 
sidered separately. This leaves a sample of 22 school districts. 

For the field work sample, we find means for log of population 
and log of enrollment of 11.614 and 10.051, corresponding to popula- 
tion and enrollment levels respectively of about 110,000 and 23,000. 
These are both substantially higher than the means for the survey 
sample. Means and medians are again nearly equal, though the dis- 
tributions for both log population and log enrollment are somewhat 
flatter for the field work sample than for the survey sample. The 
enrollment range is also narrower — from 1,362 to 650,000. See Table 
11. 
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Lew 'lork Cit> Coi;;. unity icnocl Districts 
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12.416 
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Table 10 



*Coinmunity district 11, Parkchester in The Bronx, is the only one 
in the field sample. 

These are logs of average district values. 
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The field work sites are typically more urban than the other 
sites in the survey sample • The mean proportion of the school 
district population that Is urban Is .902 • But over two-thirds 
(68.2 percent) of the districts are at least 98 percent urbaHi^aaA 
the median value is 99.8 percent urban. Only two of the 22 dls«* 
trlcts are below 60 percent urban » compared to 46 » or 24.7 parcent^ 
of the survey sample. See Tables 12 and 13. 

Proportions black and Spanish language are more similar for 
the two samples. Mean proportion black is .130 » which is somewhat 
higher than the .105 mean for the survey sample but is within one 
standard error of the estimate of the (field sample) mean. The 
shapes of the distributions for the two samples are almost iden- 
tical. Compare Table 5 and 12. Mean proportions Spanish language 
are .053 for the field work sample and .060 for the survey sample. 
The distributions are also very similar for proportion Spanish 
language across the two samples. For the field sample » proportion 
Spanish language ranges from .003 to .310» with a median of .020. 
Seven of the 22 have less than 1: percent Spanish language » and 3 
have less than 1/2 percent. See Table 12. 

Family Incomes are typically higher in the field work districts » 
with both a higher proportion over $25^000 income and a lower pro- 
portion poor» than for the survey districts as a whole ^ but the dif- 
ferences in means are very small. The distributions are much flatter 
and cover a narrower range in the field sample. Proportion of fam- 
ilies with Incomes over $259000y ^•ti^f ranges from .012 to .127 ^ has 
a mean of .051 » a median of .040 » and a kurtosis measure almost equal 
to the value for a normal distribution in the field sample. The 
corresponding values for the survey sample are 0 to .359» .046 » .036 » 
and a kurtosis measure far from that of a normal distribution (in 
the direction of greater peakedness). See Table 14. 

Representativeness 

The comparison of vector means between the survey and field %rork 
samples is a more straightforward operation than the comparison in 
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Section II between the survey sample and the population of all school 
districts. Here, we have identical data for both samples, so we 
don't have to estimate population means from other sources, and we 
know the covariances for the population. 

In testing for equality of vector means between the field work 
and survey samples, we find that the differences are not signifi- 
cant at the 5 percent level. The field sites, then, are roughly 
representative of the whole survey sample, at least on the seven 
variables considered, even though the field sites tend to be sub- 
stantially larger and more urban school districts. 

New York City 

Of the 9 community school districts in New York City that are 
included in the survey sample, one is in the field sample. That one 
is community district number 11, It is very close to the average of 
the other 8 in population and enrollment and somewhat lower in propor- 
tion black. See Table 10, 

Tables included in this Appendix provide sxsnmary descriptions 
for each of the field sites (excluding New York City) used for in- 
depth study. Tables 15-1 to 15-7 give histograms for the 7 variables 
used in Section III, The five categories from each histogram provide 
a convenient way for comparing any field site with the others in the 
sample. The categories are given for each variable for each school 
district in Table 15-8 (refer back to Tables 15-1 to 15-7 for defini- 
tions of variables and categories), 

5 

Let y « the vector of means for the survey sample, x « the 
vector of means for the field work sample, Z « the covariance matrix 
for the survey sample, N « the number of observations (in this case 

22), and p « the number of variables (in this case 7), Then 

— —1 2 

N(x-vi)* I (x-y) is distributed as x • See, e,g,, Anderson [1], 

- -1 - ^ 7 

In this case, N(x-^y)» Z (x-y) « 12.183, the ,90 point of " 12*0, 

2 ' 
and the ,95 point of X7 « 14.1. 
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